Abstract
There has been much publicity surrounding the use of machine learning technologies in self-driving cars and the challenges this presents for guaranteeing safety. These technologies are also being investigated for use in manned and unmanned aircraft. However, systems that include “learning-enabled components” (LECs) and their software implementations are not amenable to verification and certification using current methods. We have produced a demonstration of a run-time assurance architecture based on a neural network aircraft taxiing application that shows how several advanced technologies could be used to ensure safe operation. The demonstration system includes a safety architecture based on the ASTM F3269-17 standard for bounded behavior of complex systems, diverse run-time monitors of system safety, and formal synthesis of critical high-assurance components. The enhanced system demonstrates the ability of the run-time assurance architecture to maintain system safety in the presence of defects in the underlying LEC.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
ASTM F3269–17: Standard practice for methods to safely bound flight behavior of unmanned aircraft systems containing complex functions (2017)
Cofer, D., et al.: A formal approach to constructing secure air vehicle software. IEEE Comput. Mag. 51, 14–23 (2018)
DARPA: Assured Autonomy. https://www.darpa.mil/program/assured-autonomy
Denouden, T., Salay, R., Czarnecki, K., Abdelzad, V., Phan, B., Vernekar, S.: Improving reconstruction autoencoder out-of-distribution detection with mahalanobis distance (2018). CoRR, abs/1812.02765
Gacek, A., et al.: Resolute: an assurance case language for architecture models. In: HILT 2014, pp. 19–28. ACM, New York, NY, USA (2014)
Feiler, P.H., Gluch, D.P.: Model-Based Engineering with AADL: An Introduction to the SAE Architecture Analysis and Design Language, 1st edn. Addison-Wesley Professional, Boston (2012)
Kestrel Institute: APT: Automated Program Transformations (2019). https://www.kestrel.edu/home/projects/apt/
Loonwerks: AAHAA: Architecture and Analysis for High-Assurance Autonomy. http://loonwerks.com/projects/aahaa.html
RTCA DO-178C: Software considerations in airborne systems and equipment certification (2011)
Sha, L.: Using simplicity to control complexity. IEEE Softw. 18(4), 20–28 (2001)
Whalen, M.W., Gacek, A., Cofer, D., Murugesan, A., Heimdahl, M.P., Rayadurgam, S.: Your “what” is my “how”: iteration and hierarchy in system design. IEEE Softw. 30(2), 54–60 (2013)
Acknowledgments
The authors wish to thank our colleagues James Paunicka, Matthew Moser, Alex Chen, and Dragos Margineantu for their support during integration and testing on the BR&T autonomy platform. This work was funded by DARPA contract FA8750-18-C-0099. The views, opinions and/or findings expressed are those of the author and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Cofer, D. et al. (2020). Run-Time Assurance for Learning-Enabled Systems. In: Lee, R., Jha, S., Mavridou, A., Giannakopoulou, D. (eds) NASA Formal Methods. NFM 2020. Lecture Notes in Computer Science(), vol 12229. Springer, Cham. https://doi.org/10.1007/978-3-030-55754-6_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-55754-6_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-55753-9
Online ISBN: 978-3-030-55754-6
eBook Packages: Computer ScienceComputer Science (R0)